The present disclosure relates to image and video processing and, in particular, to improved image and video processing by using the characteristics of a color space.
Many modern electronic devices support exchange of video between them. In many applications, a first device captures video locally and codes it for transmission to a second device. The second device may decode the coded video and display it locally. The first device may perform pre-processing operations upon the source video to condition it for coding and/or transmission. Typical pre-processing operations include color space conversion, resizing of video, frame rate conversion of video and/or video filtering operations among others. Several coding protocols have been defined to support video coding and decoding operations. They include, for example, the MPEG-2, MPEG-4, H.263, H.264 and/or HEVC coding protocols.
The majority of digital image and video applications employ a Y′CbCr color space encoding for the representation of images, given its ability to better take into account human visual perception than other color spaces such as RGB or XYZ. Y′CbCr, also called YCbCr, Y′UV, or YUV, comprises of a luma (Y′) component, and two chroma/color components (Cb and Cr) for each pixel. The prime (′) symbol indicates the application of a transfer function on the original linear light R, G, and B signals, allowing for perceptual quantization. However, a number of other color spaces may also be used.
The components of a color space share certain relationships with each other. For example, in a YCrCb color space, the Y component may be calculated given the R, G and B components of an input signal as long as general characteristics of the video signal are known. For example, general characteristics of the video signal may include the color primaries being used, e.g. whether the signal uses BT.2020, P3D65, or BT.709 color primaries, the chroma sampling (phase) location, the color difference processes (or as are commonly called in some specifications the matrix coefficients), and the transfer functions applied to the original linear light signals, among others. Similarly, Cr and Cb components may be calculated using the Y and R, G, and B values along with the general characteristics of the video signal. Given the relationship of Cb or Cr with Y, as well as of Y with B and R respectively, limits on the value of Cb and Cr are impacted by the value of Y. If Y is known, then Cb and/or Cr are only allowed to be within a particular range of values.
However, existing algorithms tend to process each component independently, without taking into account the limitations in value that may be imposed upon certain components by previously calculated components. For example, during an encoding operation (assuming input data is provided in an N-bit integer representation, e.g. N=8) values commonly are clipped independently to stay within the valid limits of the N-bit representation, i.e. from a value of 0 up to (2{circumflex over ( )}N)−1. Further, if an encoder determines that image data corresponds to a particular limited representation, such as the standard/limited representation used for TV applications, additional clipping within that range may be performed. However, existing algorithms perform additional clipping only when the signal is fully converted back to an RGB representation. This may result in accumulation of out of range sample values, and thus introduce artifacts in the final image representation.
The inventors perceive a need for a chroma processing scheme that can operate on each component in a color space individually, and operate on subsequent components with respect to previously ascertained components, taking into account the relationships between the components to allow for more efficient processing.